

REC'D 1 2 h 


WlPO 

PCT 


Patent Office 
Canberra 


I. JONNE YABSLEY, TEAM LEADER EXAMINATION SUPPORT AND 
SALES hereby certify that annexed is a true copy of the Provisional specification 
in connection with Application No. 2002953466 for a patent by BRAMBLES 
AUSTRALIA LIMITED as filed on 20 December 2002. 



WITNESS my hand this 
FiiBi day of January 2004 



JONNE YABSLEY 

TEAM LEADER EXAMINATION 

SUPPORT AND SALES 


PRIOMTY 
DOCUMENT 

SUBMIITED OR TRANSMITTED IN 
COMPUANCE WITH RULE 1 7. 1 (a) OR (b) 


I 


10 


SOFTWARE AND METHODS FORAUTOMATED PALLET REPAIR 
Field of the Invention 

The present invention relates generally to the repair of wood^ shipment pallets, and 
specifically to an automated process for scanning pallets and identifying individual 
elements of the pallet for removal, replacement, or repair. 

Background of the luvention 


Commercial movemeat of materials typically use a wooden pallet on which the material is 
placed or secured. This pallet is typically constructed with a flat upper deck consisting of 
planks or boards of timber nailed, screwed, or glued to parallel beams known as bearers or 
stringers. Bottom boards are similarly attached to the bearors. The framework allows the 
IS insertion of the Torks" of a forklift or other machine to raise and move the pallet and its 

load of materials. There are sev^al pallet designs in use and are distinguished generally 
by place of manufacture and use. For example, pallets made and used in Australia, New 
Zealand, the United States, Canada, and Europe are all of different designs. In some 
designs, blocks are used with or in place of bearers to s^arate the top and bottom boards. 

20 

During normal use, the pallets may be dropped, overloaded, crushed or otherwise 
damaged. Damaged pallets are often returned to the pallet provider or pooler for 
inspection, repair or replacement. 

25 The inspection and decision process is cunrently done by skilled human inspectors who are 

trained to implement die specific criteria, or by automated processes which decide to repair 
or discard each damaged pallet. Using human inspectors is desirable because they can 
inspect and immediately repair each pallet at a single station. This can be done by 
presenting each damaged pallet in turn, for example, on a conveyor, such that the inspector 

30 can see pallet damage and repair it if necessary. It is undesirable because the inspection 

and repair decision is not uniform as each inspector will naturally implement repair based 
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on his or her judgement It is also undesirable from a safety point of view as accidents will 
happen in such an environment. 

Current automated systems use cameras or laser devices to collect pallet geometry and 
5 topography information, then use computer programmes to make a repair / discard decision 

by uniformly implementing tfie specific pallet criteria. This is done by first determining 
the pallet's design, e.g. Australian or European, and comparing the geometry and 
topography of the individual pallet against criteria for the pallet's design. Current systems, 
however , decide only to repair or discard each inspected pallet, that is, each pallet either 

10 does or does not pass the inspection criteria. If the pallet passes, it is placed back in 

service. If it does not pass, it is sent for repair. The repair process, however, is similar to 
the inspection process above. The pallets needing repair are sent, for example, by 
conveyor, past one or more hiunan repair stations where the repairer inspects the pallet, 
determines what needs repair and then repairs it This has the same disadvantages m 

15 uniformity and safety of the human inspection process. It is faster, however, smce the 

himian inspectors are seeing only those pallets detennined to need repair. 

Objects and Summary of the Invention 

20 It is an object to provide an automated process which not only inspects pallets and 

detennines when to repair or discard, but also identifies the pallet element or elements that 
need to be removed, replaced, or repaired, and passes this additional information to a 
human or automated repairer. 

25 It is therefore an object of the present invention to automatically inspect wooden pallets, 

identify the pallet design, compare that geometry and topography of the pallet against 
design criteria, determine when to repair or discard, identify specifically the element or 
elements needing repair and the exact nature of the repair needed. 


-2. 


It is another object of the present design to provide the repair data to an automated repair 
process, thus eliminating both human inspectors and human repairers. This makes the 
process of pallet repair faster, safer, and more uniform than current methods. 

Accordingly, a pallet is placed on an inspection station and moves under one or more 
synchronised lasers which collect geometry and topography information for the pallet. The 
geometry and topography information is normalised through scaling and rotation so that 
the rqjresentation of tlie pallet is considered to be located within a coordinate system with 
the origin at a pallet comer and the axes along the edges of the pallet The normalised 
image is reduced from three dimensions to two dimensions. The two dimensional 
representation of the pallet is analysed so that the individual elements are identified and 
located by coordinates. The pallet design is determined by the number, size, and location 
of the elements. The elements are analysed against specific criteria for the pallet's 
determined design. This includes criteria for the element alone (size, location, integrity, 
damage, missing or raised nails, etc.), inter-elemental criteria (spacing, overiap, etc.), and 
pallet design criteria (missing or superfluous elements, etc.). After analysis, if the pallet is 
determined to have passed the criteria, it is placed back in service. If the pallet is 
determined to have not passed the criteria, a list of specific repairs is generated. This list 
includes which element is to be repaired and the nature of the r^air (remove, replace, 
reattach, repair, etc.). The data comprising the list of repairs to be made accompanies the 
paUet to the repair station. In prefeired embodiments, the repair station is an automated 
repairer, for example, a robot arm using a nail gun, band or other saw, prying levers, etc., 
to implement the exact repairs determined necessary. After repair, tiie pallet is returned to 
service. 

In some embodim^ts the present invention provides coordinate outputs sufficient to 
automate, through robotic arm for example, the component repairs. 

Brief Description of the Drawings 

Figure 1 is a schematic top plan view diagram of a pallet; 

Figure 2 is a schematic top plan view diagram of an inspection station; 


Figure 3 illustrates a schematic side elevation of a pallet; and 
Figure 4 illustrates a logic flow chart of a pallet repair process. 


Detailed Description 


Figure 1 illustrates one design of a paUet 100. The paUet consists of top boards 102. 
labelled a TB 1 through 8. The top boards are supported by three horizontal bearers or 
stringers 104, 106, and 1 08. Other designs wiU have different numbers of boards, 
different sized boards, and different spacing between boards. Li the illustrated design, 
top board 1 and top board 8 are wider than the other six boards. For the purpose of this 
mvention the palette can be considered to be laid out in an x y z configuration where the 
X axis runs horizontally (with reference to Figure 1) across the bottom of the pallet 
parallel to beam or bearer 108. The y axis runs verticaUy (with reference to Figure 1) 
along the left edge of top board I. The z axis is orthogonal to both x and y. 

The present invention consists of (for example) two computer systems and a mechanical 
means for moving pallets. The first computer system is called the capture system, which 
collects infomation about the geometry and typography of the pallet. The second 
computer system is called the analysis system, which analyses the geometry and 
typography of the pallet and determines the pallet design and then using the pallet design 
specifications analyses the pallet and decides whether it is to be repaired or returned to 
service. The mechanical means moving the palette may be a conveyor belt or a robotic 
arm or other system for transporting the pallet beneath the lasers. 

Figure 2 Ulustrates one implementation 200 of the mechanical means for moving pallets. 
This comprises a conveyor belt supporting apaUet 204 and moving it in the direction of 
the arrow, left to right. The palette 204 moves under twin lasers 206 and 208. These 
lasers illuminate and sensors capture the entire width of the pallet 204. In some 
embodiments there is an overlap in the laser beams in the middle as shown by 210. As 
the pallet 204 moves under the lasers 206 and 208, information about the geometry and 
topography of a talent is captured by reflected laser light sensors and sent to the capture 


computer system 212. Such laser and sensor systems are well-known, as are the methods 
for using such lasers, to collect this information. 

The capture computer system 212 performs the following stq)s. First it collects the 

profile information of the pallet, that is, it collects the geometry and topography 

information from the lasers and sensors. The sensors return a stream of three- 
dimensional coordinates. In one implementation, each laser sensor returns 508 points for 
each scan. The lasers are synchronised so that the overlq)ping points are scanned by both 
lasers and give the same values. The scans from the two lasm are then combined to give 
a set of coordinates for the entire width of the pallet This process is repeated for each 
scan as thepaUet moves under the lasers. 

The scanned data is fdtered to give only the top surface geometry and topography. This 
accomplished by discarding any points which have a z-coordinate which is below a given 
threshold or filter line. This removes from analysis any point corresponding to a bearer 
or the bottom of the pallet or the transporting conveyor, for example. 

The laser scans are timed according to the speed of the pallet transport mechanism so that 
scans occur at regulai* distances along the length of (he pallet in the direction of 
movement Typically, this is set to scan at 1mm linear distance. 

Next, the comer points of the paUet are foimd using a 45 degree filter. This locates the 
four points at the extremities of the pallet. That is, it finds the point of minimum x 
minimum y, minimum x maximum y, maximum x minimum y, and maximum x 
maximum y. These four points determine the comers of the pallet. These are typically 
referred to as PO, PI, P2, and P3 respectively, where PO and P2 lie on the x-axis, and PO 
and P 1 lie on the y-axis. PO and P3 are diagonally, as are PI and P2. 

Next, it finds the offsets between the image origin and pallet origin to give the x and y 
offset distances of the pallet That is it calculates the size of the paUet by subtracting 
combinations of PO, PI, P2, and P3. The data is also normalised by relocating the 


coordinates so that PO lies at the origin of the coordinate system, and PI and P2 lie on the 
X- and y-axis respectively. 

At this time it also finds any protruding nail locations using Sobel or Gaussian operators. 
This process will find any point with a discontinuous z-axis value. Protruding nails or 
other features are noted for later repair. 

Next it converts the three-dimensional image to two dimensional representation of the 
pallet Fig. 3 illustrates a pallet 300 topography of the X-axis from PO to P2. The 
location of tiie top boards is identified by removing any point below a reference line 302 
drawn through the nominal centre of the board cross-section. Thus boards 304 can be 
distinguished from inter-boaid space 306, and fix)m the beam or bearer 308. 

Finally, the location of the bearer boards is detennined The bearer boards are typically 
labelled BO, B 1 , and B2. Looking in the direction the pallet is travelling along the 
conveyor, the left most or front most bearer is BO, the centre bearer is Bl and the other 
bearer is B2. The imaginary line above each bearer will be called the board filler line. 
The ofifeet measurement is the dimension that is added to the calculated error height to 
give the board fiUer line, shown as element 302 in Figure 3. The board filler lines will be 
used such that if there's a point above this line then it is assumed that their is a board in 
this location. An anay is then constructed of values 0 and 1 were zero means no board is 
present and one means a board is present Thus the array of zeros and ones for the entire 
top service of the pallet, approximately 1.2 miUion points, is constructed and passed to 
the analysis computer 214. 


The analysis computer system 214 performs the steps shown in Table I. 


TABLE 1 


1 step 

Description 

] 

The byte stream i-s used to construct a pallet image. This process uses the known 
resolution of the lasers and the captured image data stream to locate the pallet edges the 
edges of the top boards, and the exposed edges of the bearens nr .ctrinp..,^ h-Rm«: ' 

2 

A preliminary calculation to detennine the pallet type is done to identify, for example. 
USA or Australian design. If it is known that only one type of pallet will be processed 
this st^ may be omitted. For example, Australian paUets have three desiens "A5"ha<! 
- five top boards, "AS' has six top boards, and "A?" has seven top boards. 

3 

inepauet type is used to retrieve from a database of specifications soecific criteri« fnr 
the pallet t>pe. TTie "theoretical data" is compared to the pallet data, for example, the 
number of boards, to identify fee closest deteimined pallet design tvpe 

4 

1 < 

t he theoretical specific cntena are compared against die image of the pallet If any of 
these cntena fail, the paUet is marked for removal from the process and no further 
■ analysis is done. These criteria include maximum board gap, minimum board widtii, etc 

1 

Idcnuty me Mges in the pallet image. Identify the four comers of each top board. Build 
an array of edge pomts for each board. 

6 

Sort the arrays so diat the pomts are m a known order, and that the airays identify boards 
from top board 1 through the last too board. y <^ 

7 

Oieck fee comers of the boards that they roughly he on die Unes foiming the outside 

edges of the pallet. If the boards do not form a rectangle it is marked for removal fiv^m 

** A'rvunij^iv, If. lo ulcuKcu lur removal nrom 

tne process and no further analysis is done 

8 

Check that the anays representing the boards do not overly. If the boaids overlap the 
paUet is marked for removal from the process and no fiiither analysis is done 

9 

Check the first ana last airays agamst the exterior boundary points to verify that they 
represent the first and last boards in the pallet design 

10 

11 

For each array representing a top board, check the board against paUet design specific 
cntena, for example, board width, identify any notches or missing wood, ofjaffied 
edges. Mark any array which fails these criteria as a board to be removed 


Aldus pomi.au arrays are marked as either valid boards, or as boards to he r„mov.H 
t^iMmnc the arrays ror me tiist and last boards to determine if the board is in the correct 
position relative to the pallet perimeter, or whether the board needs to be adjusted If it 
is to be adjusted, the anav is changed to reflect the new board position 

13 

llic gaps between tne Doard are examined (usmg the process shown in Fig. 3) to identify 
boards which must be removed, or missing boards which must be added 

14 

^iiect all repair qjeoificabons. If none, then mark the pallet as not needing repair and it 
IS put back m service. If one or more repair specifications are found, they are collected 
and ordered. If the analysis computer is connected to an automated repair system for 
example, a robot aim, the repair instructions are sent to the repair system. If repairs are 
done manually, the repair instnictions are printed or othenvise transmitted to the pallet 
repair station to be used by the repairer. The repair specifications include oidets to 
remove and replace damaged boards, remove and reposition leading edge boanls 
remove and replace boards which have gaps larger than the design maximum ' 
instructions to remove protruding nails, etc 
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Fig. 4 illustrates the logic flow for step 13 in Table 1, the process for examining the gaps 
400. The software defines the storage arrays to hold gap values to use for eadi of the 
pn)mes402. Each gap is initialised to zero. Starting with the right hand edge of the left 
most board, the gap values are calculated for each boaid as shown in step 404. In st^ 

406 thegap values fpreachboardhavebeen^ored, SO theaverageppcanbecaM^^ 

In step 408 the average gap is compared against the criteria for gap based on the pallet 
design. If the gap is larger than the design criteria, the gap is marked as a bad gap shown 
in stq, 418. In step 420. the bad gap is examined to see whether it is large enough to fit a 
new boaid. If it is large enough to fit in a board, step 424 calculates how many boards 
will fit into the gap. That nmnber of boards is then mdicated for the repair onlers. The 
software then moves to stq, 422 to examine the next gap. However, if the decision made 

at step 420 is that the gap is not wide enough to fit in the new board and it Still excecds 

the criteria for the maximum gap. then st*,, 428 is performed. Step 428 detennines which 
boards must be removed and replaced to fix the gap, 

hi step 430 a check is made to see if one of the bounding boards is crooked. If the 
bounding boards are crooked, the offending boaid is indicated for removal and 
replacement or repositioned, and the resulting gap is re-evaluated 426. If none of the 
bounding boards are crooked 430, tiien a check is made to see if one of the boards is 
mi«mg any wood 432. If one of boards is missing wood. 432. an onler is indicated to 
remove the board and re^aluate the resulting gap in step 426. Jn step 432, if no boards 
are mrssmg any wood, then step 434 is performed. A check of the neighbouring gaps is 

made. Ifo^e of the neighbouring gaps is smaller than the other then an order is indicated 
to remove the board and re-evaluate the resulting gap step 426. If in stq, 434 the gaps 
are equal size then stq, 436 is performed, that is. the pallet is marked for manual 

inspection. 


With reference to step 408. the average gap is compared agamst the design criteria, and if 
the average gap is acceptable, a test is made to detemiine if fliere is a notch in the boatxi. 
Such a notch would give a false indication of a bad gap. The notch test is shown in steps 


410 through 416» At 410, the gap values across the notch length are added and the 
average calculated. In step 416, the calculated average is compared to the design criteria. 
If the average is greater than the design criteria and the gap is too big processing 

continues to step 418 described above. If at step 416, the average gap passes the test 

against the design criteria that a check is made in step 414 to detennine if there are more 
gap values to check. If there are more to check, the step 412 is perfoimed to add the next 
value and then subtract the first value and recalculate the average. Processing then 
continues and step 416. If at step 414 there are no more gap values to check, then it has 
been detenmned that all gaps values are acceptable and processing continues at step 422 
to move to the next gap. If there are no more gaps to test, then processing ends at step 
438. 


Returning now to stqp 426, a repair order indicates the removal of a board or the 
r^sition it. Processing then continues at 404 to recalculate the average gap. 

In this way, the pallet is examined board by board and repaus ordere are stored for later 
use or the pallet is returned to service. If the pallet needs repair, specific instnictions are 
determined for removing, replacing, repositioning boards, or to add one or more boanis, 
or to remove and replace a protruding nail. 

This provides a technical advantage over current automated pallet repair systems which 
only determme a pass-fail decision for the pallet's suitability, and no specific repair 
mstrucUons are generated. In addition, the technique of the present mventiori is sufficient 
to automate the repair process by connecting the ou^ut of the process to an automated 
repair station. Such a station could comprise a robot arm which grasps the pallet to be 
repaired, then, using a band saw and nail gun, removes and r^laces specified boards. 
The instructions to the robot ami would be to, for example, '^remove the board located 
between 22.5 cm and 40 cm fi-om the leading edge, then nail a new boaid at 22.0 cm fiom 
the leading edge." 


This same logic may be applied to the inspection of and repositioning of or replacement 
of the bearers. 
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